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ABSTRACT 

Several  versions  of  data  quantizers  are  considered  as  detection 
nonlinearities  in  a  binary  signal  detection  problem.  Randomized  and 
dithered  versions  of  these  quantizers  are  formulated  and  the  performance 
of  all  proposed  systems  is  compared  to  that  of  systems  using  some  well- 
known  detection  nonlinearities.  In  particular,  the  robustness  of  the 
quantizer  systems  in  noise  with  uncertain  statistics  is  compared  to 
that  of  the  well-known  systems  via  asymptotic  relative  efficiency. 
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1.  INTRODUCTION 

Robustness,  as  the  term  is  used  here,  refers  to  the  ability  of  a 
detection  system  to  perform  well  despite  slight  variations  in  the  noise 
statistics  from  those  used  to  design  the  system.  In  general,  for  a  set 
of  given  noise  statistics,  a  robust  detector  will  be  outperformed  by  the 
optimal  detector.  For  variations  in  these  statistics  however,  the 
optimum  detector  often  performs  poorly  when  compared  to  a  robust 
detector.  In  this  thesis  we  propose  some  detectors  involving  randomized 
data  quantization  and  dithering  and  consider  their  robustness  properties 
relative  to  other  commonly  used  systems. 

The  signal  detection  model  we  consider  here  is  a  simple  binary 
hypothesis  test  between  a  hypothesis  and  an  alternative  given  as  follows: 

H^:  ^  +  SSj^ 

H0:  xi  *  ni 

Here,  x^  is  the  i-th  observation  sample,  n^  is  noise  with  a 
probability  density  function  f(x),  Sj^  is  a  known  signal  and  9  is  a  signal- 
strength  parameter. 

The  detector  structure  to  be  considered  here  is  illustrated  in  Fig.  1. 
To  reflect  uncertainties  in  the  noise  statistics,  we  use  a  model  proposed 
by  Huber  [ 1]  as  used  by  Martin  and  Schwartz  [2]  and  hereinafter  referred 
to  as  the  mixture  model: 

Let  f(x)  be  the  noise  probability  density;  we  assume 

f(x)  ■  (l-e)0(x)  +  eh(x);  where  0  <  e  <  1;  (1.1) 

$(•)  is  the  unit  normal  density; 

h(*)  is  an  arbitrary  density;  and 

e  is  known  and  is  usually  small. 


The  clearest  comparison  of  the  two  systems  would  be  through  the 
power  and  false  alarm  rates  or  the  probability  of  error.  This  method 
quickly  becomes  intractable  for  the  problem  under  consideration  and 
is  abandoned  in  favor  of  the  asymptotic  relative  efficiency. 

The  efficacy  of  a  System  A  of  the  form  of  Fig.  1,  is  defined  as 
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and  the  performance  of  System  A  compared  with  a  System  B  also  of  the  form 
of  Fig.  1  is  given  by  the  ratio  of  their  efficacies  which,  via  the  Pitman- 
Noether  Theorem,  yields  the  asymptotic  relative  efficiency  (ARE)  (Capon 


[3]). 


ARE 


A  ,B  E. 


(1.3 


Physically,  the  ARE  represents  the  savings  in  samples  required  by  System  A 
to  achieve  the  same  power  and  false  alarm  rate  as  System  B.  Thus  if  A  is 
more  effective  than  B,  ARE.  „  >  1.  If  System  A  is  more  complicated  than  B 
then  the  ARE  should  be  considerably  larger  than  unity  to  be  practically 
useful. 

A  quantizer  is  a  nonlinear  operation  which  maps  Che  real  numbers 


to  the  real  numbers  in  the  following  manner: 


The  real  line  Is  divided  into  discrete  non-overlapping  adjacent 
segments.  A  sample  point  falling  in  one  of  these  Intervals  is  mapped 
to  a  discrete  point  corresponding  to  that  interval.  Let  t^.k  ■  l,...,m 
be  the  set  of  endpoints  of  all  the  intervals  (hereinafter 

called  "breakpoints").  Then  for  t^  >  0  Vi  and  tQ  ^  0,  the  positive  real 

m 

numbers  are  formed  by  ]R+  »  lim  U  [tj-.t^).  l*et  the  point  to  which 

t  at  k«l 
m 

each  interval  maps  be  2  0  (the  "level")  then  the  action  of  the  quantizer 

Q  on  sample  x  is  Q(x|  t  <  x  <  t,  )  ■  q,. 

k-1  K  K 

Although  not  strictly  required,  in  this  paper  the  condition  q^  >  q^ 
k  ”  l,...,m  holds  and,  in  some  cases,  q^  »  0.  Also,  with  the  exception 
of  the  dead-zone  detector,  qfc  €  [t^t^j].  Figure  2  shows  a  typical 
quantizer . 

Examples  of  quantizer  detection  systems  with  fewer  restrictions  are 
available,  see  Poor  and  Thomas  [4,5]  or  Kassam  [6]  in  particular.  In 
addition,  some  work  has  been  done  to  optimize  a  quantizer  with  a  given 
number  of  levels  by  proper  breakpoint  and  level  selection  according 
to  some  fidelity  criterion.  See  Max  [  7  ]  or  Kassam  [  6  ]  . 

Two  ad  hoc  schemes  for  modifying  quantizers  are  attempted  here  - 
randomization  and  dithering.  These  techniques  are  inspired  by  reported 


improvement  in  low-bit  digital  video  by  similar  methods  in  two  papers, 
one  by  Roberts  [  8 ]  and  another  by  Thompson  and  Sparkes  [ 9 ] . 
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Randomization  refers  to  the  perturbation  from  a  fixed  position 
of  the  quantizer  breakpoint.  This  is  achieved  in  the  following  manner: 

One  treats  the  interval  length  (tk  -  tk_^)  as  a  r« tdom  variable 
which  takes  a  value  at  each  sample  independently  of  its  value  in  other 
samples.  The  length  of  each  interval  (t^  -  t^_^)  is  given  the  value 
of  this  random  variable.  Appendix  B  clarifies  this  approach  and  gives 
the  derivation  of  the  expression  for  the  efficacy  of  this  type  of 
quantizer-detector  (Eq.  (B.4)). 

Dithering  is  performed  exactly  as  Roberts  performed  it  with  video. 

A  random  variable  uniformly  distributed  between  (-t^/2 , t^/2) ,  sample-wise 
independent  is  added  to  the  quantizer  input  and  subtracted  from  the 
output.  Figure  3  clarifies  this  process.  Appendix  C  contains  a 
derivation  of  the  efficacy  for  this  system. 

Appendix  A  is  a  derivation  of  the  efficacy  of  a  nonrandomized , 
nondithered  quantizer.  A  comparison  of  (A.9),  (B.4)  and  (C.12)  will 
show  that  the  quantizer  output  levels  have  no  bearing  on  system 
performance  (as  we  define  it  here)  except  in  the  dithered  system. 

In  the  following  sections,  specific  quantizers  and  modification 
methods  are  described  and  their  performances  compared  via  Eqs.  (A.9), 
(B.4),  (C.12),  and  (1.3). 
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2.  THE  FOUR-LEVEL  SYMMETRICAL  QUANTIZER 


This  quantizer  is  depicted  in  Fig.  4  and  is  the  simplest  quantizer 
other  than  a  sign  detector  type  or  dead-zone  quantizer  which  is  relatively 
easy  to  optimize.  Only  three  breakpoints  are  present  at  0  and  +  t.  The 
four  levels  are  spaced  at  +  q  and  +  2q. 

The  efficacy  of  this  quantizer  when  used  in  the  detector  structure 
of  Fig.  1  is  given  by 


lim 
n  -  « 


2  tf(0)  +  fml2 

3  7/6  -  F(t) 


(2.1) 


Where  f(x)  and  F(x)  are  the  density  function  and  distribution  function 
respectively,  of  the  noise. 

If  s^  is  assumed  equal  to  one  for  all  i  (the  parameter  Q  will  control 
the  actual  received  signal  strength),  then  the  efficacy  E^p  may  be 
normalized  to  the  following: 


•n  -  1  LfiP) 

4Q  3  7/6  -  F(t) 


(2.2) 


The  optimum  breakpoint  t  t  can  be  found  by  maximizing  over  t. 

This  leads  to  the  following  necessary  condition  on  t 

2  +  £<0)  +  '"opt’  '  0  (2-3) 


"hece  ''('opt1  4  df  £<*>l 
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Fig.  4.  Four-level  symmetrical  quantizer. 
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2  2 

If  f(x)  »  exp(-x  /2a  )/(*/2tt  a),  Eq.  (2.3)  is  a  transcendental 


equation  not  directly  solvable  for  t  Computer  results  were  used 

however  to  produce  the  estimate  t  _  ■  a  for  this  case. 

opt 

Randomization  of  this  quantizer  is  achieved  by  making  t  a  random 
variable  with  a  one  sided  density  over  [Q,»)  called  g(x)  and 
corresponding  distribution  function  G(x).  Then  the  efficacy  is  given  by 
(2.4),  a  special  case  of  (B.4). 

ff(0)+J‘  f(t)dG(t)  1  Z 

TWn  *  4  L - -T— - (2.4) 


7/6  -  f  F(t)dG(t) 
0 


We  consider  in  particular  the  case  where  g(x)  is  a  Rayleigh 
function  with  parameter  a  >  0: 


(x)  -  (x/a2)exp(-x2/2a2) 

With  f(x)  again  a  Gaussian  function,  becomes 


2  2  2 
(a  +  2q  ) 


\4Q  'n  2r.,  2  ^  2.2  ,  .  2  A  2.3/2. 

a  [4(0  +  a  )  -  3a(a  +  a  )  J 

This  can  be  maximized  over  or  to  find  a 

opt 


opt 


This  yields 


\4Q| 


Of 'Of 


opt 
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=  .7807002/a2 


(2.5) 
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The  performances  of  Che  two  quantizers  were  compared  using  Che 
mixture  model  of  (1.1)  with  h(x)  a  Gaussian  density  with  variance  a The 
results  of  this  comparison  are  shown  graphically  in  Fig.  5.  It  is  seen 
that  for  large  contamination  the  randomized  quantizer  performs  only 
slightly  better.  The  trend  for  small  values  of  a  seems  to  indicate  a 
large  performance  improvement  but  the  small  range  over  which  this  occurs 
may  not  compensate  for  the  extra  complexity  of  the  system  necessary  to 
achieve  this  improvement. 

It  is  worth  noting  again  that  the  actual  level  value,  q,  does  not 
appear  in  the  efficacy  expression.  The  output  of  the  quantizer  indicates 
into  which  interval  the  received  sample  falls.  This  information  alone  is 
all  that  is  necessary  to  make  a  decision  between  HQ  and  H^,  therefore 
the  actual  value  of  the  level  is  arbitrary. 
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3.  THE  DEAD -ZONE  QUANTIZER 


The  dead-zone  or  null-zone  quantizer,  as  taken  here,  is  a  three 
level  symmetrical  quantizer  with  output  levels  0,  +  q  and  breakpoints 
+  t.  This  type  of  detector  was  studied  in  a  different  context  by  Kassam 
and  Thomas  [ 10]  .  Their  use  of  it  is  different  enough,  however,  that  the 
results  are  not  comparable  with  those  given  here.  Figure  6  illustrates 
the  dead-zone  quantizer. 

The  efficacy  of  this  quantizer  is  given  by  (3.1)  for  a  symmetrical 
noise  density  function  f(x). 


(3.1) 


An  optimum  breakpoint  tQpt  can  be  found  for  a  given  density  f  by 

2 

maximizing  over  t.  If  f  is  Gaussian  with  variance  a  ,  then  topt  «  ct. 

Randomization  is  achieved  here  by  letting  the  breakpoint  t  be  a 
random  variable  with  some  distribution  G(x)  (and  corresponding  density 
g(x)).  The  efficacy  of  the  randomized  dead-zone  quantizer  then  becomes: 

“  2 

2[f  f  (t)dG(t)j 

\l)Z  "  (3.2) 

f  [l-F  (t)]dG(t) 

0 

Three  possible  density  functions  were  examined,  each  with  a  single 
parameter  which  could  be  adjusted  to  maximize  (3.2).  These  are  listed 
below 

1.  Single-Sided  Exponential 
g(x)  -  V  expl-Yx]  'u(x) 


(3.3) 
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2.  Rayleigh 


g(x)  -  (x/a2)exp[-x2/(2a2)] *u(x) 


(3.4) 


3.  Triangular 


g(x) 


x/r 


(3.5) 


0  <  x  <  r 

-x/r'"  +  2/r  r  <  x  <  2r 

0  elsewhere 

Since  Che  nominal  noise  in  the  mixture  model  of  (1.1)  used  throughout 

this  paper  is  unit  variance  Gaussian  noise,  the  three  parameters  were  each 

2 

optimized  for  f(x)  a  Gaussian  with  variance  a  . 

Under  these  conditions  then,  the  following  results  are  achieved: 

V, 


'opt 

a  .  -  otljl 
opt 

r  «  0 
opt 


(3.6) 


Clearly  VQpt  and  rQpt  lead  to  degenerate  forms  in  (3.3)  and  (3.5). 

An  arbitrary  value  of  y  was  used  to  get  some  results  but  the  use  of  the 
triangular  density  was  abandoned  at  this  point.  Performance  measurements 
are  based  upon  an  exponential  contamination  noise,  that  is,  in  reference 
to  the  previously  described  mixture  model,  we  have 


h ( x )'  ■  O/2)exp[-0  |  x | ]  ,  6  >  0.  (3.7) 

Figure  7  shows  the  performance  of  the  nonrandomized  dead-zone 
quantizer  compared  with  that  of  the  4-level  quantizer  described  above. 

Three  values  of  e  (.1,  .2  and  .3)  are  used  and  indicate  the  trend  of  the 
ARE  for  this  case.  The  dead-zone  quantizer  performs  only  slightly  better 
for  e  >  .2  and  only  in  a  narrow  range  of  0. 
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The  small  amount  of  improvement  and  the  narrow  region  over  which  it 
occurs  are  reasons  enough  to  exclude  the  dead-zone  quantizer  as  a 
replacement  for  the  4-level  quantizer.  Coupled  with  these  reasons  is 
the  question  of  the  validity  of  the  mixture  model  as  a  representation  of 
uncertainty  in  the  noise  model  when  e  becomes  large.  The  mixture  model 
here  is  meant  to  represent  noise  that  is  primarily  Gaussian.  The  detector 
structure  is  designed  with  this  assumption  and  its  performance  monitored  as 
the  noise  deviates  from  unit  variance  Gaussian,  the  amount  of  deviation 
being  represented  by  e  and  the  "direction"  of  deviation  depending  upon  the 
function  used  for  h(x).  Thus  if  e  is  large  (e  a  .5  is  probably  a  liberal 
definition  of  large)  one  would  do  better  to  abandon  the  mixture  model  and 
examine  some  othev  technique  for  detector  design. 

Does  randomization  improve  the  performance?  Figure  8  shows  the  ARE 
of  the  exponentially  randomized  dead-zone  (RDZE)  compared  with  the 
nonrandomized  version.  The  value  used  for  y  is  y  *  1A/2tt.  This  is  an 
arbitrary  value  and  the  results  are  not  good.  Since  y  ■  -H»  maximizes 
(3.2),  these  results  are  not  unexpected. 

Figure  9  shows  the  change  in  performance  from  using  a  Rayleigh 
distributed  breakpoint.  Once  again,  the  improvement  is  slight  and 
occurs  only  for  small  8  and  large  e. 

Dithering  also  provides  a  possible  modification  of  the  dead-zone 
quantizer.  This  process  again  is  best  summarized  by  Fig.  3.  In  the 
application  of  dithering  to  this  case  the  random  variable  d^  (hereafter 
called  the  dither  signal)  is  given  a  uniform  distribution  in  the  interval 
C-c,c]  where  c  is  a  positive  constant.  Two  cases  are  considered  here: 

1)  c  -  t,  2)  c  -  t/2. 


Fig.  8.  Exponentially  randomized  dead-zone  compared  with  dead-zone  quantizer. 
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Letting  G(x)  and  g(x)  be  the  distribution  and  density  respectively 
of  the  dither  signal  we  have  the  following  expressions  for  the  efficacy 
(3.8)  and  the  normalized  efficacy  (3.10)  of  the  dithered  dead-zone 


quantizer. 


^DDZ*  lim  n 


(<£sj)2[/  {f(d  +c)+f(d  -c)}dG(di)]2 

_ -«0 _ 

q^Cl-J  {F(d-K:)-F(d-c)}dG(di)]-2^s2[r  (f (d£+c)+f (d1-c)}dG(di)] 


-  [c^si(l-J'  (F(di+c)+F(d.-c)}dG(di))]2  +Es2  VarCd^) 


(3.8) 


where  all  summations  are  over  i  from  1  to  n. 


Under  the  assumption  that  d^  and  the  noise  are  sample-wise  independent 


we  make  the  following  assumption: 


n(2  s  ) 

lim  - Jr  “  0 

n  -  «  (E  sp 


(3.9) 


Under  (3.9)  then,  the  next  to  last  term  in  the  denominator  of  (3.8) 
becomes  zero  in  the  limit.  Since  this  term  is  subtracted,  and  the  term 
itself  is  positive,  the  denominator  value  will  only  be  increased  and  at 
worst,  the  normalized  efficacy  will  be  lower  than  if  the  term  were 
maintained.  Thus,  rePr@sents  a  worst  case  efficacy  in  those  cases 

where  (3.9)  does  not  strictly  hold. 

Cf  £f (d+c)+f (d-c)JdG(d)] 2 

^DDZ  *  m  ~  •  2  (3.10) 

1-J*  {F(d+c)-F(d-c)}dG(d)  -  |  J  {f  (d+c)+f  (d-c)}dG(d)  +  ~ 

-»  q  -«  3q 

•jf  4 

This  assumption  would  be  true,  for  example  if  -  (-1)  ;  i  ■  l,2,...,n. 


Note  that  dithering  has  increased  the  complexity  of  the  efficacy 
expression  and  has  brought  the  level  value  q  into  play.  We  may  now  find 


a  qopt  which  maximizes  the  denominator  of  (3.10). 

Using  a  Gaussian  distribution  for  F(x),  qQpt  was  found  for  Cases 


1  and  2: 

Case  1: 


►  _  /2c^.  ,  ,2a  -a  2  .  2c.  l-i. 

*  qopt  “  ^  3  ^  c  a  "  2  ^ 


(3.11) 


Case  2:  c  *  t/2 

qopt  *  (c2+a2){$(  2j)**(|“))]  +  ^ZzCexpC^J-expC1^)]  ) 

<v/2tt  8a  8a 

(3.12) 

The  fact  that  q  enters  into  these  calculations  at  all  is  directly 

attributed  to  the  dithering  process,  therefore  one  should  not  be  surprised 

that  different  densities  for  the  dither  signal  alter  the  nature  of  q 

nopt 

In  Case  1,  it  is  only  necessary  for  the  received  signal  to  vary  from  0 
for  the  dither  signal  to  affect  the  quantizer  output  whereas  in  Case  2, 
the  input  may  be  anywhere  inside  the  interval  [-t/2, t/2]  before  the 
dither  comes  into  play. 

The  actual  derivation  of  Eqs.  (3.11)  and  (3.12)  is  tedious  but 
straightforward  and  will  not  be  repeated  here. 

A  check  of  the  efficacy  values  for  Case  1  and  Case  2  using  only 
Gaussian  noise  (no  mixture  model)  for  various  values  of  a  showed  that 
Case  1  was  uniformly  better.  The  dead-zone  quantizer  itself  was  optimized 
with  t  ■  a .  Table  1  summarizes  the  results  for  a  few  values  of  a. 
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TABLE  1 

Efficacy  of  Dithered  Dead-Zone  Quantizer 


a 

Case  1 

Case  : 

.5 

2.56 

.30 

1.0 

.73 

.09 

1.5 

.28 

.04 

2.0 

.12 

.02 

2.5 

.05 

.02 

Based  on  these  results  Case  2  was  abandoned  in  favor  of  Case  1. 
Henceforth  all  references  to  the  dithered  dead-zone  quantizer  will  imply 
the  use  of  the  Case  1  density  for  the  dither  signal. 

Results  were  also  obtained  for  values  of  t  other  than  t  *  ct  and 
showed  that  the  efficacy  is  generally  better  for  a  value  of  t  slightly 
less  than  a .  For  this  reason,  results  obtained  with  the  mixture  model 
use  t  =  .9. 

Both  the  Gaussian  mixture  model  (h(x)  -  N(0,<72))  and  the  exponential 
mixture  model  (h(x)  as  in  Eq.  (3.7))  were  used  to  evaluate  the  dithered 
dead-zone  quantizer.  Figures  10  through  16  summarize  the  results  . 

In  addition  to  the  comparison  with  the  nondithered,  nonrandomized  dead- 
zone,  Figs.  12,  15  and  16  show  the  comparison  to  the  sign  detector. 

These  curves  serve  to  relate  the  performance  of  the  system  to  a  well-known 
robust  detector.  Equation  (3.13)  gives  the  efficacy  of  the  sign  detector. 

^SD  "  4  f2(0)  (3.13) 


Dithered  dead-zone  compared  with  dead-zone  quantizer 


Fig.  15.  Dithered  dead-*one  compared  with  sign  detector 
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Comparing  Figs.  10  and  13  or  Figs.  11  and  14,  improvement  with 
dithering  is  obtained  for  small  values  of  a  and  large  values  of  g. 

(Note  that  these  refer  to  two  different  mixture  models.)  Both  of  these 
cases  represent  densities  h(x)  which  are  becoming  more  sharply  peaked  at  0. 
That  is,  as  a  decreases  in  the  Gaussian  mixture  model,  and  as  B  increases 
in  the  exponential  mixture  model,  h(x)  becomes  more  sharply  peaked  at  0. 
This  improvement  is  clearly  much  greater  for  the  exponential  contamination 
however  and  extends  for  all  g  >  1*3.  The  improvement  for  Gaussian 
contamination  is  not  only  less  marked  but  exists  only  for  o  <  1  and 
larger  values  of  e  which  again  brings  to  mind  the  question  of  the 
validity  of  the  results  as  a  basis  for  detector  design. 

Comparison  of  Figs.  12  and  15  or  16  show  that,  under  Gaussian 
contamination,  the  dithered  system  performs  considerably  better  than  a 
sign  detector  over  a  wide  range  of  both  a  and  e,  whereas  under  exponential 
contamination  the  dithered  is  worse  by  far  except  for  very  small  e.  Even 
for  this  small  e  the  ARE^^  ^  m  1.2  which  is  not  a  significant  improvement 
if  one  considers  the  additional  complexity  of  the  dithered  system. 

In  summary,  dithering  seem.*  to  produce  unpredictable  results. 
Performance  with  sharply  peaked  contamination  densities  is  improved 
but  that  improvement  is  sensitive  to  the  density  function  and  may  not 
be  significant  when  the  dithered  dead-zone  is  compared  to  some  other 
detection  system  (e.g.,  the  sign  detector). 


In  an  attempt  to  make  the  performance  of  the  dithered  dead-zone 
less  sensitive  to  the  contamination  density,  a  sign  nonlinearity  can  be 
placed  in  line  following  the  correlator.  For  brevity,  this  new 
detector  is  referred  to  as  the  dithered  sign  detector  although  it  is  not 
really  a  dithered  version  of  the  sign  detector.  Figure  17  will  clarify 
the  structure  of  this  system. 

Keeping  all  notation  from  the  preceding  discussion  of  the  dead-zone 
quantizer,  we  have  the  efficacy  of  the  dithered  sign  detector: 

“  o 

TTdsd  -  CJ*  (f(t-d)  +  f(-t-d)]dG(d)]  (3.14) 

-a 

In  order  to  compare  this  with  the  previous  system  all  parameters  of 
the  dead-zone  quantizer  are  maintained.  That  is,  G(d)  represents  a 
uniform  distribution  over  [-t,t],  t  =  .9  and  q  is  given  by  (3.11). 

The  results  of  comparing  the  dithered  sign  detector  to  both  the  dead- 
zone  quantizer  and  the  sign  detector  in  Gaussian  and  exponential 
contaminated  noise  are  presented  in  Figs.  18  through  21.  These  graphs 
show  almost  uniform  degradation  in  performance.  The  only  improvement 
appears  in  the  comparison  to  the  sign  detector  in  Fig.  21.  It  appears 
here  that,  for  decreasing  9  and  for  e  >  .4,  the  gp  increases 

drastically.  The  importance  of  this  result  is  once  again  undermined  by 
the  large  value  and  narrow  range  of  e  over  which  this  occurs. 
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It  appears  from  these  results  that  a  dithered  dead-zone  quantizer 
may  be  useful  in  comparison  with  an  ordinary  dead-zone  quantizer  for 
severe  contamination  of  Gaussian  noise  with  exponentially  peaked  noise 
densities.  This  appears  to  be  restrictive  in  the  light  of  the  reason  for 
proposing  the  dithered  system  -  namely  to  improve  robustness.  If 
robustness  indicates  relative  insensitivity  to  changes  in  the  noise  it 
seems  wasteful  to  use  a  system  of  increased  complexity  to  gain  only 
moderate  performance  improvement  over  a  restricted  range  of  noise 
contamination. 
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4.  THE  2m- LEVEL  QUANTIZER  AS  AN  APPROXIMATION  TO  THE  LIMITER  CORRELATOR 

Our  LasC  investigaCion  of  a  quantizer-detection  scheme  is  possibly 
the  most  general.  Here  we  compare  a  quantizer  with  2m  levels  (m  is  an 
integer)  with  the  limiter-correlator  as  proposed  by  Martin  and  Schwartz 
[2  ].  The  motivation  here  is  that  the  quantizer  (which  is  uniform  and 
symmetrical)  appears  as  a  discrete  approximation  to  the  limiter  correlator 
and  represents  the  approximation  which  would  be  achieved  by  a  digital 
computer  implementation  of  the  detector. 

First  we  will  reiterate  the  development  of  the  limiter-correlator  as 
in  Martin  and  Schwartz.  The  problem  is  one  of  deciding  between  a  hypo¬ 
thesis  and  alternative  Hq  and  H^  as  described  in  the  Introduction.  The 
noise  density  f(.)  is  assumed  Co  belong  to  the  family  of  densities 

9  -  {f(x)|f(x)  -  (l-e)cp(x)+eh(x):h(->  €JC,  0  <  e  <  l)  (4.1) 

where  cp(x)  denotes  the  unit  variance  Gaussian  density  with  zero  mean,  X 
is  the  class  of  all  symmetric  density  functions  satisfying  the  following 
regularity  condition: 

Regularity  Condition:  Let  1(9)  =  J 4(x)h(x-9)dx  where  £(•)  is  a 
bounded  function  and  h(*)  €X.  Then  1(9)  has  one  continuous  derivative 
given  by 

I' (9)  *  JX(x)  (dh(x-9)/d9)dx 

Since  cp(x)  satisfies  this  condition  as  well  as  h(x),  then  $  also 
satisfies  the  regularity  condition. 

Note  that  the  other  mixture  models  used  throughout  this  thesis 
(with  h(x)  ■  N(0,a  )  and  h(x)  -  0 /2 ) exp [-0 | x] ])  also  meet  this  condition. 
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Lee  &  denote  the  set  of  all  randomized  test  functions  0  ■  0(X) 
where  X  ■  (x^,...,^)  is  the  set  of  N  observations.  That  is,  0(X) 
represents  the  probability  of  choosing  as  true  given  that  the 
observation  is  X. 

The  power  function  P^(9]f)  of  0  6 &  is  the  probability  of  choosing 
given  that  is  true  using  the  randomized  test  function  0.  This  is 
given  by 

00 (9 1  f)  -  Eg  [0 (X) i  f ]  f  €  i?  (4.2) 

The  false- alarm  probability  is 

““p0(olf>  (4*3) 


The  problem  is  to  find  a  locally  most  powerful  test  0  and  a  least- favorable 
density  f  such  that  the  following  conditions  for  the  power  are  true: 


where 


^  col  f)  <  ^(o|l) 

0$(o|  f)  >  f^(0|f) 
p£<ol  •>  -  (d/d9)p?(e|  -)je 


(4.4) 

(4.5) 


If  $  is  indeed  the  locally  most  powerful  test  for  f,  then  $  and  f  form  a 
saddle-point  solution  in  terms  of  local  power  with  constrained  false  alarm 
probability  and  (4.6)  and  (4.7)  below  are  satisfied. 

inf  p’(o|f)  -  P,(o|f)  -  sup  p'(o|l)  (4.6) 

*  *  0€$  0 

Of  -  Pj(0lf)  <  p^(0if)  -  5  (4.7) 

Now  f  and  0  must  be  specified  for  each  e  such  that  (4.6)  and  (4.7)  hold 
asymptotically  over  a  restricted  range  of  a  bounded  away  from  zero  and 
dependent  on  e . 


— -’JV’-W** 
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The  leasc-£avorable  density  £  is  given  by 


or,  since 


f(x)  - 

(l-e)"^exp[-g(x)] 

x2/2  I x|  <  K 

g(x)  - 

K|x|  -  K  /2  [x|  >  K 

f(x)  - 

(l-s)cp(x)  +  «h(x) 

(4.8) 


h(x) 


(4.9) 


(2ne2)"^(l-e)[exp(-K|x|+K2/2)-exp(-x2/2)]  |x|  >K 


In  addition,  if  h(x)  is  to  be  a  valid  density  function  (i.e.  if 

00 

J  h(x)dx  *  1),  then  K  satisfies 


J  cp(x)dx  +  2(cp(K)/K)  -  (1-e) 
-K 


(4.10) 


Now  for  a  noise  density  given  by  f,  the  density  for  an  observation,  x^, 


given  is  true,  is 

f0(xi)  =  f(x£  -  Ss^  (4.11) 

According  to  the  generalized  Neyman- Pear son  Lenina  [15]  then,  the  locally 
most  powerful  test  $  of  size  a  is  given  by 


1  Tn(X)  >  c 


?N(X>  “'<(  a  Th(X)  -  c 


(4.12) 


0  Tn(X)  <  c 


where  0  <  a  <  1 


N 


and 


T„(X)  -  2  s.i  (x, ;  -K,K) 


(4.13) 


Here,  y;A,B)  indicates  a  soft  limiter  characteristic  given  by  (4.14) 
below 
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4(y;A,B) 


B 

y 

A 


y  >  b 
A  <  y  <  B 
y  <  A 


(4.14) 


This  characteristic  can  be  derived  from  the  derivative  of  the  probability 
ratio  test  using  f(x)  above,  i.e.  the  expression  of  (4.12)  is  equivalent  to 


JL  n  £(V9sl} 
ae  11 


i=l  f(xt)  1 8*0 


<  T 


(4.15) 


where  r  is  a  threshold  chosen  to  achieve  a  desired  probability  of  false 
alarm  a.  Constant  terms  are  includes  as  a  new  threshold  c. 

The  resulting  detector  structure  has  been  dubbed  the  limiter-correlator 
and  is  illustrated  in  Fig.  22.  Note  that  the  limiter  is  symmetrical  and 
that  large  e  increases  the  limiting  action  (i.e.,  reduces  K). 

To  conclude  the  description  of  the  correlator- limiter  we  include 
Theorem  2  of  [ 2  ] . 

Define 

Y  *  (A  +  eK^/A  (4.16) 

m 

where  K  -  , 

A  -  (l-c)J  x  9(x)dx  +  2k  $(-K)  (4.17) 

-K 

Since  K  is  a  function  of  e ,  Y  is  also,  therefore  Y  *  Y  (e)  and  a  is  then 

in  m  m 

also  a  function  of  e  given  in  terras  of  Y  : 

m 

«(*)  -  l-4>([21ogy  /Y2  -  1)]*)  (4.18) 

m  m 


-* ‘.‘■xryt •V’  . K 
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Theorem  :  Let  f  be  the  least  favorable  density  (29)  and  «L  the  corre- 
-  N 

sponding  locally  most  powerful  test  given  by  (33)  and  (34).  If  5  >  a(e), 

0  <  c  <  1,  then  asymptotically 

inf  pi  (o[  f )  «  (o|f)  *  sup  P^  (0|  f ) 

and 

h  <°lf>  ^  H  «>l*>  "  5 

A  comparison  of  the  limiter  characteristic  of  Fig.  22  and  the  typical 
quantizer  of  Fig.  2  reveals  the  similarities  between  the  two.  If  *  K, 
then  the  quantizer  may  be  considered  a  discrete  approximation  to  the  limiter 
characteristic.  As  the  number  of  quantizer  steps  grows,  i.e.  as  m  is 
increased,  and  as  long  as  =  K,  the  steps  become  finer  and  the  quantizer 
becomes  a  closer  approximation  to  the  limiter.  From  experience  with  the 
quantization  of  analog  signals  in  the  ordinary  A/D,  D/A  case  of  signal 
processing,  one  expects  more  steps  to  create  an  improvement  in  performance, 
upper  bounded  by  the  performance  of  the  continuous  (or  piecewise-continuous) 
system  to  which  the  quantizer  is  an  approximation. 

In  standard  data  quantization,  the  comparison  between  a  system  function 
and  its  quantized  version  is  mean-squared-error  [7],  absolute-mean-error  [11], 
or  some  more  general  error  criterion  [  12,13].  These  methods  do  not  always  lead 
to  the  best  solution  when  one  considers  a  quantizer  in  a  binary  decision 
system,  however,  and  some  other  criteria  may  give  better  results,  e.g. 
distance  measures  |14] . 

To  reiterate  then,  the  best  detection  quantizer  may  not  be  merely  the 


least-mean-square-error  approximation  to  the  analog  optimum  systems.  In 


light  of  this  fact,  the  following  questions  can  be  asked  concerning  the 
performance  of  the  2m- level  uniform  quantizer  as  an  approximation  to  the 
limiter-correlator : 

1.  What  is  the  ARE  of  the  quantizer  compared  to  the 
limiter  (AREQ>LC)? 

2.  How  does  randomization  affect  the  AR£a  . 

Q,LC 

3.  How  does  dithering  affect  the  ARE  ? 

Q,  LC 

Implicit  in  theses  queries  is  the  effect  of  m  on  the  ARE.  _. 

Q,LC 

Why,  if  this  quantizer  may  be  suboptimal  as  a  detector  are  we  worried 
about  its  performance  compared  to  the  analog  system?  First  of  all,  signal 
processing  of  any  complexity  is  quite  likely  to  be  carried  out  digitally, 
implying  a  quantization  of  data.  Secondly  this  quantization  is  likely 
initially  to  be  of  the  uniform,  2m-level  type  since  this  is  the  nature  of 
most  commercially  available  A/D  converters.  Thus  the  need  for  a  nonuniform 
quantizer  might  overly  complicate  a  system  for  little  gain  in  terms  of  detector 
performance.  Thus  the  answers  to  the  three  questions  above  could  well 
indicate  the  economy  of  constructing  a  more  complicated  quantizer. 

The  efficacy  of  the  limiter-correlator  is  given  by 


2K2  J  f (n)dn  +  J  n2f (n)dn 


and  that  of  the  2m-level  uniform  quantizer  by  (see  Appendix  A) 


2[Z  (2k-l)( f ( (k-l)t)  -  f(kt)}  +  (2m-l) f ( (m-l)t) ]‘ 


E  (2k-l)  iF(kt)  -  F ( (k—  1 ) t ) }  +  (2m-l)‘f  l-F((k-l)t)} 
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If  q^  »  K,  then  t  *  K/(m-l/2)  to  maintain  the  similarity  between  the 
two  detectors. 

The  results  here  are  multidimensional.  ARE's  are  calculated  and 
plotted  for  both  the  Gaussian  contaminated  and  the  exponential  contami¬ 
nated  noise  with  m  *  3,4  and  S.  In  addition  this  is  carried  out  for  the 
least- favorable  noise  density.  All  of  these  results  use  e  and  K  as 
parameters  to  form  a  set  of  four  curves  for  each  m  in  each  noise  case. 
Plotting  the  ARE  as  a  function  of  e  was  not  feasible  due  to  the  compli¬ 
cated  relationship  between  e  and  K  as  evidenced  by  (4.10).  The  four  paired 
values  of  e  and  K  are  listed  in  Table  2  below. 


Table  2 


e 

.01 

.02 

.05 

.10 

K 

1.95 

1.72 

1.40 

1.14 

Figures  23  through  25  show  the  quantizer  performance  in  the  Gaussian 

contaminated  noise.  Surprisingly  ARE_  >  1  for  all  cases  considered. 

x  > 

Two  trends  are  clearly  evident:  the  quantizer  performance  improves  over 
the  limiter-correlator  for  more  severe  mixtures  (e  increasing),  and  as  m 
increases,  the  quantizer  performance  decreases.  Also,  at  first  glance, 
one  would  expect  the  performance  for  all  mixtures  to  be  equal  at  a  ■  1, 
however,  it  must  be  remembered  that  K  changes  with  c  so  that  even  though 
m  remains  the  same,*  the  quantizers  are  not  equivalent  (nor  are  the  limiters) 
for  different  e.  The  improvement  obtained  here  is  significant  and  is  quite 
probably  worth  the  implementation  of  a  quantizer. 


#4* i***1  "■ 


& 


1.14 


48 


The  next  question  is  how  does  randomization  affect  the  ARE?  In  this 
case  the  intervals  [t^_^ ,t^)  are  treated  as  random  variables  all  with 
identical  uniform  densities.  To  be  more  precise,  we  define  u  to  be  the 
length  of  the  interval  [t^^  ,  a  constant  for  all  values  of  k:  u  «  t^-t^  1 

We  then  make  u  a  random  variable  with  a  uniform  density  function  over  (0,c). 
The  effect  is  a  quantizer  which  collapses  and  expands  as  u  takes  on  smaller 
and  larger  values  respectively.  Note  the  level  values  q^  are  not  changed. 

If  u  *  0,  we  have  a  sign  detector  with  output  levels  +  q^.  If  u  =  c,  the 

» 0  >  <1^  >  •  •  • » 

t,  =  kc . 
k 


q^}  with  breakpoints 


output  levels  consist  of  the  set  {-^.....-q^ 


For  these  results,  c  =  2t^  where  t^  is  the  first  breakpoint  of  the 
corresponding  non-randomized  quantizer,  i.e.  c  *  2K/(m-l/2). 

Given  the  above  density  function  for  u  we  have  an  expression  for  the 
efficacy  of  the  randomized  2m- level  quantizer  given  by  (aee  Appendix  B). 


2  m- 1  c  c 

%  [  Z  (2k-l)  J  [f((k-l)x)  -  f(kx)}dx+  (2m- 1)  J  f((m-l)x)dx] 
k=l  o  o 


(4.21) 


Figures  26,  27,  and  28  illustrate  the  performance  for  m  *  3,4  and  5 
respectively.  Obviously  randomizing  the  interval  is  not  a  technique 

V 

useful  for  improving  performance  in  this  case.  Once  again  we  notice  a 
tendency  for  the  AREon  to  improve  for  small  a  indicating  some  improvement 
by  randomizing  for  peaked  contamination  noise.  However  for  this  to  compete 
with  the  nonrandomized  quantizer,  a  must  be  very  small. 


Fig.  26.  Comparison  of  randomized  quantizer  with  limiter-correlator 
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Figures  29,  30,  and  31  are  the  performance  curves  of  the  non- 
randomized  quantizer  for  n  *  3,4,5  in  exponential  contaminated  noise. 

Figures  32,  33,  and  34  are  the  corresponding  curves  for  the  randomized 
quantizer.  The  observations  here  are  much  the  same  as  for  the  Gaussian 
contamination  case  -  the  quantizer  performance  is  markedly  better  than  the 
limiter  and  randomization  ruins  this  performance.  In  this  case,  randomization 
produces  uniformly  worse  results  with  no  promise  of  an  increase  in  ARE 

RQ  i  LC 

for  any  g  or  e. 

Randomization  produces  such  terrible  results  that  one  would  be  led  to 
expect  a  similar  degradation  with  dithering. 

In  calculating  the  efficacy  of  the  dithered  system,  the  variance  given 
Hq  must  be  found.  This  is  given  by 

n  3  A 

Var[TN(X  )U01  -  E  »*( Var +  d^H^  -  2Etd.- +  d^l^]}  (4.22) 

i*l 

where  n^  *  noise  samples 

d^  -  dither  signal  samples 
Q(x)  ■  output  of  quantizer  with  input  x 
(Note:  Appendix  C  contains  a  general  solution  for  this  term). 

Here  d^  has  a  uniform  density  function  over  (-t/2,  t/2). 

The  last  term  in  (4.22)  is  the  correlation  between  d^  and  the  quantizer 
output.  The  effect  of  d^  is  to  cause  Q(n^  +  d^  to  change  by  only  a  single 
step  value  up  or  down  from  what  Q(n^)  would  be ,  if  d^  causes  a  change  at 
all.  Thus,  although  d^  and  Q(n^  +■  d^)  are  not  independent  or  uncorrelated. 


we  can  see  that  the  correlation  is  small  and  positive.  Since  the  calculation 
of  this  term  becomes  very  complicated  we  will  take  it  to  be  zero.  This 


Comparison  of  randomized  quantizer  with  limiter-correlator 


huftw  wi  Y  ■a>ri,,ji'  latvaqfla. 
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establishes  a  lower  bound  on  the  efficacy  of  the  dithered  quantizer 
since  inclusion  of  the  term  can  only  increase  the  value  of  . 

Under  this  assumption  the  efficacy  of  the  dithered  quantizer  is 
given  by 


m 


\>Q 


2[  r  (2k-l){2F((k-i)t)  -F((k-f)t)  -F((k+i)}]2 

k=l _ j _ _ _ z 

m  ,  kt  2 

t  E  (2k-l)z  [  J  [F(zi+|)  -F(z.-  j))dZi  +  ] 

kml  (k-l)t  12ql 


(4.23) 


where  z^  *  n^  +  d^.  Note  that  once  again,  dithering  brings  the  output 
level  q  into  play. 

Figure  35  shows  the  best  ARE  curve  obtained  with  the  exponential 

LL 

contaminated  noise.  It  appears  that  no  advantage  is  gained  by  dithering 

here.  As  e  increases,  so  does  the  ARE,  but  again  the  values  at  which  any 

performance  gain  appears  are  too  large  for  a  valid  noise  model. 

As  a  final  check  of  the  validity  of  the  results  for  the  2m-level 

quantizer  we  examine  the  AREn  Tr  when  the  noise  density  is  given  by  (4.8), 

the  least  favorable  density.  Since  the  limiter-correlator  is  the  optimum 

detector  for  this  case,  we  expect  ARE  r  <  1  for  all  e.  The  values 

Q,  Lc 

obtained  are  summarized  in  Table  3. 


Table  3 


ARE 


Q.LC 


e 

ra  *  3 

m  ■  4 

m  «  5 

.01 

.952 

.975 

.985 

CM 

O 

• 

.962 

.980 

.988 

.05 

.975 

.987 

.992 

10 


983 


.991 


.995 
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This  reinforces  the  optimality  of  the  limiter  correlator  for  the 
least-favorable  density  as  well  as  verifying  the  validity  of  previous 
results  since  the  same  general  equations  for  efficacy  were  used  throughout 
this  section.  Also  we  see  that,  as  the  number  of  steps  increases,  the 
performance  of  the  quantizer  approaches  that  of  the  limiter-correlator. 

The  performance  also  increases  with  increasing  e  which  is  consistent 
with  earlier  findings.  Thus  it  appears  that  in  cases  where  the  limiter- 
correlator  is  optimal,  our  intuition  is  correct  in  assuming  that  increasing 
the  number  of  steps  will  improve  the  effectiveness  of  a  quantizer  used  to 
replace  the  limiter. 

The  answers  to  the  three  questions  posed  earlier  can  now  be  stated. 

A  quantizer  may  be  used  to  some  advantage  in  place  of  the  limiter.  Problems 
arise  however  in  trying  to  design  the  quantizer.  If  the  noise  is  close  to 
the  least  favorable  noise  in  some  sense,  we  would  do  well  to  provide  as 
many  steps  as  possible  to  approach  the  performance  of  the  limiter  as  closely 
as  possible.  If  on  the  other  hand,  the  noise  is  quite  different  (e.g. 
Gaussian  or  exponential  contamination)  then  the  quantizer  should  have 
relatively  few  steps  and  we  can  expect  greatly  improved  performance.  In 
either  case,  of  course,  there  is  also  the  problem  of  choosing  an  adequate 
mixture  model. 

As  far  as  randomization  and  dithering  go,  it  appears  that  both  lead 
to  either  uniformly  poor  results  or  results  which  are  beneficial  under  such 
narrow  conditions  that  the  question  of  robustness  is  moot. 

The  best  that  can  be  done  then,  is  to  choose  a  noise  model  (for  the 
mixture  model)  and  to  determine  for  each  choice  whether  or  not  the  quantizer 
replacement  for  the  limiter  is  practical.  There  does  not  appear  to  be  any 
rule  of  thumb  concerning  this  choice. 
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5.  SUMMARY 

We  have  examined  quantizers  in  three  different  forms:  the  four- level 
symmetrical  uniform  quantizer,  the  three-level  or  dead-zone  quantizer 
and  the  more  general  2m-level  symmetrical  uniform  quantizer.  For  all 
three  types  the  performance  of  the  quantizer  as  a  signal  processor  in  a 
binary  hypothesis  testing  scheme  was  compared  with  several  more  widely 
used  detector  nonlinearities  known  for  their  performance  stability  with 
respect  to  slight  noise  changes. 

The  object  of  these  calculations  was  to  observe  how,  if  at  all,  the 
quantizers  performed  in  terms  of  this  stability,  called  robustness.  A 
contaminated  noise  mixture  and  the  asymptotic  relative  efficiency  (ARE) 
were  used  to  measure  this  robustness  relative  to  that  of  the  afore¬ 
mentioned  well  known  non linearities  (the  sign  detector  and  the  limiter- 
correlator)  . 

In  addition,  two  techniques  of  randomizing  the  quantizer  breakpoints 
as  well  as  a  more  general  technique  of  randomization  called  dithering 
were  introduced  and  defined.  These  techniques  were  applied  to  all  three 
of  the  above  quantizers  and  their  effects  upon  the  ARE  were  observed. 

The  results  of  these  calculations  were  presented  in  either  graphs 


or  tables. 
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6 .  CONCLUSION 

It  is  difficult  to  draw  any  general  conclusions  from  these  results. 

It  appears  that  the  decision  on  whether  or  not  to  use  a  quantizer  must 
be  based  on  an  analysis  of  the  problem  at  hand.  The  regions  where  good 
performance  is  achieved  are  usually  quite  narrow  and  this  reflects  a  neces¬ 
sarily  more  detailed  knowledge  of  the  noise  than  may  be  available  when  one  is 
concerned  with  robust  systems.  At  the  other  extreme,  good  performance 
is  frequently  achieved  for  very  high  values  of  e ,  at  which  point  non- 
parametric  systems  might  be  better  applied. 

The  possible  exception  to  this  general  trend  is  the  nonrandom  quantizer 
as  a  replacement  for  the  limiter  in  a  limiter-correlator  detector.  Here 
it  appears  that  there  may  be  a  large  class  of  mixture  models  where  a  low 
level  number  quantizer  easily  outperforms  the  limiter.  Again  it  is 
necessary  to  consider  each  model  separately  since  no  rule  of  thumb  is 
obvious . 

Finally,  the  one  uniform  result  of  this  work  has  been  that  neither 
randomization  nor  dithering  significantly  improves  the  efficacy  of  any  of 
these  quantizers.  In  fact,  the  performance  is  usually  far  worse.  As  was 
stated  in  the  Introduction,  the  motivation  for  testing  these  techniques 
was  the  success  found  in  applying  similar  methods  to  picture  processing. 

The  difference  is  in  the  way  that  the  output  is  produced.  The  decision  system 
must  of  course  produce  a  hard  decision  -  Hq  or  -  and  its  performance  is 
measurable  in  hard  numbers  -  probability  of  error,  false  alarm  rate,  etc* 

A  video  signal  on  the  other  hand  produces  an  output  which  is  further 
processed  by  the  human  mind  and  is  subject  to  many  psychological  and 
physical  effects. 


t  The  difference  between  the  two  processes  seem  great  enough  to  state 
that  no  real  comparison  is  possible,  at  least  without  an  accurate  model 
of  the  human  visual  process. 

In  short  then,  as  far  as  this  work  has  shown,  we  may  conclude  that  the 
techniques  of  randomization  and  dithering  appear  to  offer  little  in  the 
way  of  improving  quantized  decision  systems. 
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APPENDIX  A 

Derivation  of  the  Expression  of  the  Efficacy  of  a  Quantizer 

The  following  development  applies  to  the  use  of  a  quantizer  as 
illustrated  in  Fig.  2  in  a  detection  system  similar  to  Fig.  1. 

The  system  produces  the  statistic  T(x^)  given  below: 

T(xi)  -  QCx^ 

where 

Q(xt)  *  %  ; 


Xi  €  ttk-l,Ck) 


(A.  1) 


(A. 2) 


From  (1.2)  we  then  obtain  the  general  expression  for  the  efficacy 

n 


■  lim 
*  n~ 


t  @E[  Z  S.Q(x  )|H.]/&0) 
i-1  1  l 


6-0 


(A.  3) 


n  Varf  Z  s.Q(x  )|h0] 
i*l 


As  in  the  preceding  work  we  will  let  f(x)  and  F(x)  represent  the  proba¬ 
bility  density  function  and  distribution  function  respectively.  Assume 
that  f(-x)  *  f(x),  (i.e.  f(x)  is  an  even  function). 

We  will  now  use  (A. 2)  and  evaluate  the  numerator  of  (A. 3) 

Since  the  received  signal  has  sample-wise  independent  values: 

n  n  n 

E[  Z  s  Q(x  )|H.J  -  Z  s  E[Q(x  )[H  ]  -  Z  s  E[Q(n  +0s  )] 
i-1  1  1  1  i-1  1  11  i-1  1 

■  Vi  •  jlvt'W-'Vi-’V1  -qk{F(-tk.1-e,1) 

i-1  k-1 


A  A 

where  t  *  +  »,  -t  -  -  00  . 
m  m 


n 

(aE[^r^siQ<xi)|H1]/ae)j^ 
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I  S 
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i 


m 

’  1  Si 
k-1  1 


+  ’kf£<-tk-i-9si)-t<-V9si»ll9.0 


n 

-21 

i-l 


•  i  ‘M't'W  '  £<‘k» 

k-1 


Similarly  we  find  the  variance  under  H^: 


n  n 

Var[  E  s  Q(x  )|hq]  *  Var[  E  •iQ(ni)l 
i-l  i-l 


Again  using  sample-wise  independence  and  noting  that  E[Q(n^)] 

n  n  j  n 

Var [  E  a  Q(n  .)]  *  E  s  E[Q  (n.)] 
i-l  1  1  i-l 


Now 

E  s?E(Q2(n  )]  •  E  s*  E  [q^{ F(t.  )-F(t  ) }  +  q^C F(-t 

i-l  1  1  i-l  1  k-1 

and  noting  F(-x)  -  1-F(x) 


Var [  E  s  Q(x  )1hq] 
i-l 


n  2  n  2 

l  I  s‘  E  q:tF(t  )  -  F(t  )] 
i-l  1  k-1  K  K  1 


(A. 4) 

-  0  we  have: 

(A.  5) 

k-i>-F<-tk»J 

(*.6) 


Using  (A. 4)  and  (A. 6)  in  (A. 3)  yields  the  efficacy. 


4<  E  sj)2  {  Z  <lk[f(tk.1)  -f(tk)l}2 

i-1  k-1 

E„  *  lim  — . .  . .  -  — . 

Q  n  *  m  2 

"  2nE  s  E  qktF(tk>  "  F^k-1)] 
i-1  1  k-1  K  K  K  L 


If  we  again  let  s^  -  1  and  Cake  the  limit,  we  obtain  the  expression  for 
the  normalized  efficacy. 


2[  Z  qkff ‘ 

\  ■  -v— -  (A-8 

r  qk[F(£k)  -  F(t  >] 
k-1 

,  t  4.. 

m 

If  we  assume  the  quantizer  is  uniform  such  that  q^  -  (2k-l)q^ * (2k-l)q 


where 


and  fc.  -  kt  then 
k 


2{  Z  (2k-l) [f ( (k-l)t)  -  f(kt)]}i 
k-1 

V~ - ^ 

E  <2k-ir[F(kt)  -  F((k-l)t)] 
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APPENDIX  B 


Derivation  of  the  Expression  of  the  Efficacy  of  a  Randomized 

Breakpoint  Quantizer 


Here  we  will  develop  an  expression  similar  to  (A. 8)  for  the  quantizer 
of  Fig.  2  where  the  breakpoints  are  a  function  of  a  random  variable  with 
some  probability  density  g(x)  and  distribution  G(x). 


Specifically,  we  consider  all  breakpoints  t^>  k-l,...,m,  to  be 


functions  ofasingle  variable  t  such  that  t^  -  c^t  where  {c^,  k*l,...,m} 


is  a  set  of  m  positive  constants.  For  the  uniform  quantizer,  cfc  *  k  and 


t  »  t^.  Since  the  results  of  this  paper  concern  only  the  uniform  quantizer 


we  consider  only  this  case  in  our  development  here. 


Proceeding  as  in  Appendix  A,  using  (1.2),  (A. 1)  and  (A. 2)  we  can 

* 


derive  an  expression  similar  to  (A. 3), 


n 


ERQ  *  lim 


C(i>  /  El  Z  slQ(xt)|H1,t]dG(t)/b9}^ 
i=l _ _ 


(B.l) 


n  J*  Var[  Z  stQ(x  )  (Hq,  t]dG(t) 
-«□  i-1 


Assuming  the  necessary  conditions,  we  take  the  derivative  operation  inside 
the  integral  and  proceed  as  in  Appendix  A  to  arrive  at  the  expression  for 
the  efficacy 


11  .2.2,  m 


E__  *  lim 
RQ  _  _  n  „  m 


4(  Z  s‘r{  Z  qk  J  (f((k-l)t)  -  f(kt)]dG(t)]i 
i-1  k-1  -« 


(B.2) 


2n  Z  s‘  Z  J*  tF(kt)  -  F((k-l)t)]dG(t) 

i-1  k-1  -08 


where 


f(mt)  -  0,  F(mt)  -  1. 


Note  that  the  sample-wise  independence  of  the  random  breakpoints  has  been 
incorporated  implicitly  in  (B.l). 


*t> 


J 
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The  normalized  expression  is  then  given  by 


m 

2{  Z  qk  J  {f((k-l)t)  -  f(kt)]dG(t)}2 
k*l 


'RQ 


m 


(B.3) 


Z  qk  J  [F(kt>  -  F ( (k-l)t) ]dG(t) 


k-1 


Furthermore,  since  we  are  referring  to  the  uniform  quantizer,  qk *  (2k-l)q1 
(2k-l)q.  Using  this  in  (B.3)  we  get  a  final,  normalized  expression  for 
the  efficacy  in  which  the  output  level  q  does  not  appear 

m 


2{  Z  (2k-l)  f  [f((k-l)t)  -  f (kt)]dG(t)}i 
k»l  -• 

lRQ  m  2  ® 
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(B.4) 


APPENDIX  C 


Derivation  of  the  Expression  for  the  Efficacy  of  a  Dithered  Quantizer 

Herein  we  consider  the  dithered  quantizer  as  illustrated  by  Fig.  3. 

The  dither  signal  d^  is  considered1  to  be  a  random  variable,  the  value  of 

'  \ 

which  is  independent  from  sample  to  sample.  The  probability  density  and 
distribution  functions  are  given  by  g(x)  and  G(x)  respectively. 

The  test  statistic  is  given  by 

=  Q(xi  +  dt)  -  dL  (C.: 

where  Q(-)  is  described  by  (A. 2). 

We  will  consider  only  the  case  of  uniform  quantization  so  that 
qk  *  (2k-l)q  and  t  *  kt,  tQ  *  0.  Then  (1.2)  becomes 

i*l  “  *  ‘  w 

Following  Appendix  A,  we  consider  first  the  numerator  of  (C.2) 
n  n 

EfiVi<Q<Xi  +  di)  "  V^l1  “.S  siE[Q(ni  +  di+9si)  -di] 

In  all  cases  considered  here,  E[d^]  ■  0,  therefore  we  have  under  this 
assumption 

n  n 

EtiZ1Si(Q(xi  +  di)  '  di)lHlJ  "iZ1  «iBI<l(Il1  +  di  +  0s1)J  (C.3 

We  now  define  z^  ■  n^  +  d^  and  make  the  further  assumption  that  g(x)  is  an 
even  function,  symmetric  about  x-0.  Then,  with  the  same  noise  density 
f  as  in  Appendix  A,  we  can  find  the  density  f  of  the  random  variable  z,. 


E  *  lim 
DQ  _ 
n-*°° 
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f(zj  -  J  f(n  )g(z.-n  )dn 

-CO 

Let  FCz^)  be  the  corresponding  distribution  function  and  note  that 
f(z^)  is  an  even,  symmetric  function.  Now  (C.3)  becomes 


(C.4) 


It  It  111 

E[  £  s  (Q(x  +d  )-d  )|HJ  -  £  s  z  [q.[F(t  -0s  ) -F(t.  0s.)) 

i*l  1  1  1  1  K  K  1  K-l  1 

•  "ktF(-ck-i-9’i>-f<-V9si)5l 


n  m 


q  £  s  •  £  (2k-l)[F(kt-0s.)  -F((k-l)t)-0s.) 
i=l  1  k-1  1  1 


F-  (k-l)t  -  0si)  +F(-kt-0si)] 


where  F(mt)  =  i. 


n  2  m 


(dE[  £  s  T(x  )|Hj/d0)  =  q  £  sf  Z  (2k-l)  [f  (  (k-l)t-0s . )  -  f  (kt-0s, ) 

i*l  1  1  1  0=0  i=l  1k=l  1  i 


f(-kt-0Si)  +  f(-(k-l)t-0Sl)] 


n  ,  m  _ 

2q  £  s‘  Z  (2k-l)[f((k-l)t)  -  f(kt)] 
i-1  k=l 


(C.5) 


f (mt)  •  0. 


where 


72 


Next  we  consider  the  variance  term. 
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and,  because  of  independence  between  samples. 
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Recalling  that  E[d^]  -  0,  (C.6)  may  be  simplified: 
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where,  similarly  to  (A. 6) 
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The  last  term  of  (C.7)  can  be  evaluated  as  follows: 
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Finally  the  efficacy,  using  (C.7),  (C.5),  and  (C.2)  is 
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and,  normalized  as  before  with  unit  signal, 
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